Consolidating Industrial Small Files Using Robust Graph Clustering

Small file management is widely encountered in industrial areas. Consolidating small files can benefit the performance of data system. Many existing consolidation solutions fail to realize importance a proper schema. Therefore, they use very primitive and ineffective schemas. In this paper, we focus on proposing an effective robust Unlike most that only historical workload, consider workload uncertainty issue propose graph-clustering-based solution more future. To do this, introduce optimization, mathematical model provides theoretical support for solving issues. Then, demonstrate robustness schema be achieved using graph clustering algorithm with duplication mechanism. Since leads redundancy, parameter control redundancy We also two algorithms estimate automatically. Experimental results both synthetic real-life sets show effectiveness our algorithm.